Simulating Remotely Sensed Imagery for Classification Evaluation

The remote sensing community has long been active in developing, evaluating, and comparing different classification algorithms using a variety of remotely sensed imagery. As an integral component of image classification, accuracy assessment is usually conducted to evaluate the agreement between the classified map and the corresponding reference data. However, current accuracy assessment practices are limited by the difficulties in obtaining high quality reference data and the lack of spatial representation of classification uncertainties. To overcome these limitations, we developed a simulation approach to obtaining the desired reference data for better evaluation of classification algorithms. The simulation approach involves three components: 1) a real image scene, 2) a reference map, and 3) a simulated image scene. The real image scene is assumed as a random realization of a spectral probability model governed by an unknown underlying process, which is defined as the ground truth of the classification of the image scene. The reference map represents a reasonable estimate of the unknown process that generates the real image scene. The simulated image scene is generated as a random realization of the spectral probability model governed by the estimated process represented in the reference map. Specifically, an initial simulated image is firstly generated by independently sampling based on probability distributions estimated from the real image scene and reference map. Then, the initial simulated image is iteratively perturbed using simulated annealing to create a final simulated image which has similar spectral, spatial, and textual properties to the real image scene. The simulation approach was applied to a Landsat TM image scene and promising results were achieved.